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The  central  topic  of  this  project  was  the  risk-averse  formulation  of  multistage  optimization 
problems  containing  uncertainty.  The  key  issue  is  developing  time-consistent  plans  of  action  in 
situations  when  uncertainty  is  revealed  at  more  than  one  point  in  time.  Time  consistency  is 
essentially  automatic  for  models  that  are  completely  risk-neutral  or  maximally  risk  averse,  but 
becomes  problematic  in  intermediate  cases. 


1.  Computing  global  solutions  of  bilevel  risk  programs 

We  experimented  extensively  with  solving  relatively  small  but  realistic  multistage  stochastic 
programming  instances  using  time-inconsistent  risk  measures.  Although  we  use  objective 
functions  that  are  not  time  consistent,  we  achieve  model  time  consistency  by  using  multilevel 
optimization  constraints.  In  the  first  years  of  the  project,  we  reformulated  these  multilevel 
models  as  MPEC's  (Mathematical  Programs  with  Equilibrium  Constraints)  and  found  local 
solutions  using  the  FOQO  Newton-barrier  solver.  In  the  last  years,  we  further  reformulated  these 
MPEC  models  into  forms  recognizable  by  mixed-integer  programming  (MIP)  solvers  such  as 
GuRoBi  and  CPFEX.  To  do  this,  we  used  two  techniques,  one  with  auxiliary  binary  variables 
and  another  using  specially  adapted  special-ordered- set  (SOS)  constraints.  We  began  our  study 
with  a  class  of  simple  supply-chain-based  applications,  using  both  mean- semideviation  risk 
measures  and  a  blended-CVaR  risk  measures  (convex  combinations  of  expected  value  and 
CVaR).  We  obtained  the  best  run  times  using  GuRoBi  on  SOS  reformulations.  In  our 
experiments,  GuRoBi  able  to  find  globally  optimal  solutions  within  minutes,  except  for  one 
blended-CVaR  instance.  Although  solution  takes  far  longer  than  similar  models  with  time- 
consistent  objective  functions,  the  run  times  are  still  acceptable,  except  for  the  one  difficult 
instance.  For  higher  levels  of  risk  aversion,  the  identified  solutions  were  also  significantly 
different  from  those  obtained  using  time-consistent  objective  functions.  These  results  indicate 
that  our  multilevel  optimization  modeling  approach  is  likely  to  have  practical  applications.  In 
the  last  months  of  the  project,  we  began  working  with  a  multistage  portfolio  application.  We 
also  obtained  data  for  a  hydropower  system  planning  application. 

We  submitted  a  jounal  paper  containing  most  of  these  results,  “Multilevel  Optimization 
Modeling  for  Risk- Averse  Stochastic  Programming,”  by  Jonathan  Eckstein,  Deniz  Eskandani, 
and  Jingnan  Fan.  We  have  heard  informally  that  this  paper  has  been  conditionally  accepted  to  the 
INFORMS  Journal  on  Computing. 
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2.  Theoretical  results  on  time  consistency  of  models 

We  developed  a  formal  abstract  theory  of  time  consistency  of  multistage  systems  of  stochastic 
optimization  models,  where  a  system  of  models  consists  of  an  optimization  model  for  each  node 
of  a  scenario  tree.  We  proved  several  theoretical  results  relating  this  form  of  consistency  to 
existing  notions  of  time  consistency  of  risk  measures,  and  showing  that  our  proposed  multilivel 
optimization  modeling  scheme  produces  time-consistent  systems  of  models  regardless  of  the 
objective  functions  used. 

This  work  is  included  in  the  same  paper  described  above,  submitted  to  the  INFORMS  Journal  on 
Computing. 


3.  Complexity  Theory 

We  proved  that  the  simplest  subclasses  of  the  kinds  of  models  we  propose  are  NP-hard.  In 
particular,  we  proved  that  applying  our  techniques  to  three-stage  problems  with  linear  objective 
functions  and  constraints,  and  using  the  most  common  risk  measures  such  as  mean- 
semideviation  or  CVaR  still  produces  an  NT- hard  problem  class.  This  work  is  included  in  the 
same  paper  described  above,  submitted  to  the  INFORMS  Journal  on  Computing ,  and  also 
appears  in  several  online  working  papers. 


4.  Time-consistent  approximations  of  time-inconsistent  models 

With  the  goal  of  eventually  creating  specialized  algorithms  able  to  exploit  the  special  structure  of 
multistage  stochastic  programming  problems  with  time-incosistent  risk  measures,  we  developed 
new  techniques  which  iteratively  construct  time-consistent  approximations  to  time-inconsistent 
formulations.  Each  approximation  can  be  efficiently  solved  by  a  specialized  decomposition 
method.  The  solution  obtained  is  then  used  to  refine  the  approximation.  Repeating  this  process  a 
number  of  times  results  in  computable  tight  time-consistent  upper  bounds  on  the  optimal  value  of 
the  original  problem. 

We  developed  two  versions  of  this  approximate  method:  non-parametric  and  parametric.  The 
non-parametric  method  finds  the  best  bound  among  all  time-consistent  models,  while  the 
parametric  method  considers  only  models  from  a  specific  family  of  risk  measures,  allowing  for 
easy  representation  of  the  approximations.  We  tested  both  methods  extensively  and  established 
their  efficiency. 

To  obtain  universal  bounds,  that  is,  bounds  that  are  valid  for  all  possible  decisions,  not  only  the 
optimal  one,  we  developed  a  specialized  combinatorial  method.  It  is  capable  of  handling  linear 
models  with  specific  parametric  families  of  risk  measures.  The  method  is  computationally 
intensive.  We  compared  it  to  our  decomposition  methods  for  bounding  the  optimal  value.  In 
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summary,  the  bounds  obtained  by  all  methods  are  tight,  and  although  the  non-universal  bounds 
are  not  guaranteed  to  hold  away  from  the  optimal  point,  they  are  of  very  good  quality. 

We  validated  our  procedures  by  testing  them  on  a  real-world  portfolio  optimization  problem. 

These  results  are  presented  in  the  paper  “Time-Consistent  Approximations  of  Risk-Averse 
Multistage  Stochastic  Optimization  Problems,”  by  Tsvetan  Asamov  and  Andrzej  Ruszczynski, 
which  has  been  published  online  in  Mathematical  Programming  (with  print  version  in  process). 


5.  Rolling-horizon  applications 

We  experimented  with  application  of  dynamic  measures  of  risk  to  portfolio  management  on  a 
rolling-horizon  basis.  We  compared  first-  and  higher-order  semideviation  risk  measures,  and 
confirmed  that  the  use  of  higher-order  dynamic  risk  measures  gave  superior  portfolio 
performance. 

We  presented  these  results  in  the  paper  “Two-Stage  Portfolio  Optimization  with  Higher-Order 
Conditional  Measures  of  Risk”  by  Sitki  Gulten  and  Andrzej  Ruszczynski,  which  has  been 
published  online  in  Annals  of  Operations  Research  (with  print  version  in  process). 


6.  Risk-averse  control  of  Markov  systems 

Markov  control  models  are  more  appropriate  for  decision  problems  with  a  long  or  perhaps 
infinite  time  horizon.  We  refined  our  earlier  theory  of  Markov  risk  measures  to  accommodate 
undiscounted  problems,  such  as  risk-averse  stochastic  shortest  paths.  The  resulting  theory,  in  the 
form  of  risk-averse  dynamic  programming  equations,  fully  characterizes  the  set  of  optimal 
solutions  and  provides  insight  into  its  properties.  Moreover,  we  have  proposed  and  investigate 
several  new  numerical  methods  for  solving  risk-averse  dynamic  programming  models:  value 
iteration,  policy  iteration,  and  both  Newton  and  convex  programming  approaches  to  policy 
evaluation.  We  found  that  policy  iteration  with  convex  programming  for  policy  evaluation  is  the 
most  efficient  method,  with  Newton  policy  evaluation  nearly  as  good. 

This  work  is  included  in  paper  by  Ozlem  Cavus  and  Andrzej  Ruszczynski,  “Computational 
Methods  for  Risk- Averse  Undiscounted  Transient  Markov  Models,”  Operations  Research  62(2) 
(2014) 401-417. 


7.  Development  of  the  theory  of  risk-averse  dynamic  optimization  problems  for  transient 
models. 

We  formulated  the  concept  of  a  risk-transient  model  and  developed  corresponding  dynamic 
programming  equations  for  risk-averse  optimal  control.  In  this  setting,  we  demonstrated  that 
randomized  control  may  better  optimize  risk  than  deterministic  control.  As  special  cases,  we 
solved  several  examples  of  risk-averse  optimal  stopping  and  stochastic  shortest  path  models. 
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This  work  is  included  in  the  paper  by  Ozlem  Cavus  and  Andrzej  Ruszczynski,  “Risk- Averse 
Control  of  Undiscounted  Transient  Markov  Models,”  SIAM  Journal  on  Control  and 
Optimization  52(6)  (2014)  3935-3966,  and  the  aforementioned  paper  by  Ozlem  Cavu§  and 
Andrzej  Ruszczynski  in  Operations  Research. 


8.  Process-based  risk  measures 

We  created  a  new,  refined  theory  of  process-based  risk  measures,  suitable  for  use  with  Markov 
and  non-Markov  control  models.  We  also  developed  a  corresponding  theory  of  time 
consistency.  Our  main  results  in  this  area  are  derived  needing  to  measure  risk  only  for  random 
sequences  that  can  actually  occur  in  the  controlled  system.  As  a  result,  one  can  deduce  a  much 
more  refined  structure  for  the  risk  measures  than  in  more  general  settings. 

The  results  are  included  in  a  manuscript  “Process-Based  Risk  Measures  for  Observable  and 
Partially  Observable  Discrete-Time  Controlled  Systems,”  by  Jingnan  Fan  and  Andrzej 
Ruszczynski,  which  has  been  submitted  for  publication. 


We  acknowledged  AFOSR  support  in  all  submitted  manuscripts. 
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